Wherefore, what is claimed is: 

^Ctq^ A computeKrimplemented face recognition process for identifying a 

person depicted in an inpu\image, comprising using a computer to perform the 
5 following process actions: 

creating a database of a plurality of model image characterizations, 
each of which represents the facte of a known person that it is desired to identify 
in the input image as well as the rerson's face pose; 

comparing a region depicting the face of a person, which has been 
10 extracted from said input image and (tharacterized in a manner similar to the 
plurality of model images, to the plurality of model image characterizations; 

ascertaining which of the plurality of model image characterizations 
most closely matches the similarly characterized input image region; and 

designating the input image region to be the person associated 
15 with the most closely matching model image^characterization if a degree of 
similarity exceeds a prescribed threshold. 

2. The process of Claim 1 , further comprising a process action of 
specifying that the person designated as corresponding to the input image 

2 0 region has the face pose associated with\the most closely matching model 
image characterization. 

3. The process of Claim 1 , furthjir Comprising a process action for 
designating the input image region to be an unknown person if the degree of 

25 similarity between the characterized inputiregion and the most closely matching 
model image characterization does not exceed the prescribed threshold. 

4. The process of Claim 2, wherein the comparing, ascertaining, 
designating and specifying process act/ons comprises the actions of: 
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training a neural netwc&rk ensemble to identify the person 
associated with the characterized inpW image region and their face pose; and 

employing the neural network ensemble to identify the person 
associated with the characterized input f^age region, and their face pose. 



. neural network ensemble cori\prises an action of preparing each model image 



The process ^ Claim 4, wherein the process action for training the 



characterization from a model miage depicting the face of a known person that it 
is desired to identify in the input\mage by, 
10 extracting the portion of the model image depicting said face, 

normalizing the extracted portion of the model image by resizing it 
to a prescribed scale if not already at the prescribed scale and adjusting the 
region so that the eye locations of Vn\ depicted subject fall within a prescribed 
area, and 

15 cropping the extracted poVtion of the model image by eliminating 

unneeded portions of the image not specifically depicting part of the face of the 
subject to create a model face image. 



6. The process of Clam 5, whertein the process action for training the 
20 neural network ensemble further comprises actions for: 

categorizing the model face images by assigning each to one of a 
set of pose ranges into which its associatep face pose falls; 
for each pose range, 

choosing a prescribed number of the model face images of 
25 each person being modeled which have b^en assigned to the selected pose 
range, 

concatenating each a the chosen model face images to 
create a respective dimensional column vector (DCV) for each, 



computing a covarian 



e matrix from the DCVs, 
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calculating eigenvectors and corresponding eigenvalues 

from the covariance matrix, 

ranking the Eigenvalues in descending order, 
identifying a prescribed number of the top eigenvalues, 
using the eigenvectors corresponding to the identified 

eigenvalues to form the rows of a qasis vector matrix (BVM) for the pose range; 

and 

multiplying eacfti DCV by each BVM to produce a set of PCA 
coefficient vectors for each model face image. 

7. The process of Claim el wherein the neural network ensemble 
employed comprises a plurality of facft recognition neural networks each of 
which has input and output units and is dedicated to a particular pose range, 
said face recognition neural networks output units being in communication with 
input units of a fusing neural network wmich has at least enough output units to 



allow a different output to represent eaci 



of the pose ranges, and wherein the process action of training the neural 



network ensemble further comprises the 
for each face recognition n 



person it is desired to identify at each 



actions of: 

ural network, inputting, one at a time. 



each of the PCA coefficient vectors associated with the pose range of the face 
recognition neural network into the inputp of the network until the outputs of the 
network stabilize; 

initializing the fusing neural network for training; 

for each DCV, simultaneously inputting the PCA coefficient vectors 
generated from the DCV into the respective face recognition neural network 
associated the vector's particular pose range group until all the PCA coefficient 
vectors of every DCV have been inp/it, and repeating until the outputs of the 
fusing neural network stabilize; anc 

for each DCV, simultaneously inputting the PCA coefficient vectors 
generated from the DCV into the respective face recognition neural network 
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associated the vector's particularpose range group and assigning the active 
output of the fusing neural network as corresponding to the particular person 
and pose associated with the mod^ image used to create the set of PCA 
coefficient vectors. 



10 



15 



20 



8. The process of Claim 7, wherein the process action of employing 
the neural network ensemble to identif^A the person depicted in the input image 
face region and the pose associated wilh the face of the identified person, 
comprises the actions of: 

preparing the face region (fxtracted from an input image by 
normalizing and cropping the extracted legions, wherein said normalizing 
comprises resizing the extracted face region to the same prescribed scale if not 
already at the prescribed scale and adjusting the region so that the eye locations 



bribed area, and wherein the cropping 
of the image not specifically depicting 
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Of the depicted subject fall within a pres 
comprises eliminated unneeded portior 
part of the face of the subject; 

concatenating the preparjbd face region to create a DCV; 

multiplying the DCV by each BVM to produce a set of PCA 
coefficient vectors for the extracted face region; 

inputting each PCA coefficient vector in the set of PCA coefficient 
vectors into the respective face recognition neural network associated that 
vector's particular pose range group! and 

identifying the active unit of the output of the fusing neural network 
and designating the person and pose previously assigned to that unit as the 
person and pose associated with me extracted face region. 
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9. The process of Claim 3. wherein the process action for designating 
the input image region to be an uifiknown person comprises the actions of: 
training the neural inetwork ensemble to identify the person 
associated with the characterizedlinput image region to be an unknown person if 
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it does not match any of the modlpl image characterization to a prescribed 
degree; and 

employing the neurah network ensemble to identify the person 
associated with the characterized imput image region to be an unknown person if 
it does not match any of the model i\nage characterization to the prescribed 
degree. 
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10. A face recognition syste 
input image, comprising: 

a general purpose compi 



for identifying a person depicted in an 



ing device; and 



a computer program comprising program modules executable by 
the computing device, wherein the computing device is directed by the program 
modules of the computer program to, 

capture model imades, each of which depicts at least one 



rpgions within the model images, each of 
n that it is desired to identify in the input 



person of known identity, ^ 

locate and extract 
which depicts the face of a known pers^ 
image, 

determine the fac^ pose for each of the face regions 
extracted from the model images, 

categorize each ffece region by assigning each to one of a 
set of pose ranges into which its associated face pose falls, 

capture said input image, 

locate and extract regions within the input images, each of 
which depicts the face of a person, and 

for each face region extracted from the input image, 
comparing the face region tb the face regions assigned to each pose range to 
identify the person depicted in thejinput image face region. 
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1 1 . The system oAciaim 1 0, wherein the program module for 
comparing comprises sub-modules for: 

assessing the degree of similarity between the input image face 
region and the face regions assigned to each pose range; 

ascertaining whicfb face region assigned to the pose ranges 
exhibits the greatest degree of similarity to the face region extracted from the 
input image; and 

designating the faceVegion extracted from the input image to be 
the person depicted in the face region ascertained to exhibit the greatest degree 
of similarity to the input image face ftegion. 



15 
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12. The system of Claim 1 1\ wherein the program module for 
comparing further comprises a sub-module for specifying that the face region 
extracted from the input image has a flp^BH^Iing within the pose range, to which 
the face region ascertained to exhibit the greatest degree of similarity to the 
input image face region was assigned. 



1 3. The system of Claim 1 2, 
designating the face region extracted 
person if the degree of similarity betwejen 
face regions assigned to each pose rafige 



jrther comprising a program module for 
from the input image to be an unknown 
BD the input image face region and the 
do not exceed a prescribed threshold. 



14. The system of Claim 10, wherein the program module for 
comparing comprises sub-modules for: 

training a neural netwo k ensemble to identify the person depicted 
in the input image face region and \hu pose associated with the face of the 
identified person; and 

employing the neural n Jtwork ensemble to identify the person 
depicted in the input image face regi Dn and the pose associated with the face of 
the identified person. 
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15. The system 6f Claim 14, wherein the sub-module for training the 
neural network ensemble cdmprises a sub-module for preparing each face 
region extracted from said model images by normalizing and cropping the 
extracted regions, wherein salM normalizing comprises resizing each extracted 
face region to the same prescriped scale if not already at the prescribed scale 
and adjusting each region so that the eye locations of the depicted subject fall 
within the same prescribed area,\and wherein said cropping comprises 
eliminating unneeded portions of\the image not specifically depicting part of the 



1 0 face of the subject. 



16. The system of Clam 15, wherein the sub-module for training the 
neural network ensemble further comprises sub-modules for: 

(a) selecting a previously unselected one of the set of pose 

15 ranges; 

(b) choosing a prescribed number of the prepared face images 
of each person being modeled which h^ve been assigned to the selected pose 
range; 

(c) concatenating each bf the chosen prepared face images to 
20 create a respective dimensional column N^ector (DCV) for each; 

(d) computing a covariarfce matrix from the DCVs; 

(e) calculating eigenvectors and corresponding eigenvalues 
from the covariance matrix; 

(f) ranking the eigenvalues in descending order; 
25 (g) identifying a prescribeil number of the top eigenvalues; 

(h) using the eigenvectorslcorresponding to the identified 
eigenvalues to form the rows of a basis vector matrix (BVM) for the selected 
pose range; 

(i) repeating actions (a) thr^Dugh (h) for each remaining pose 

30 range; 
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(j) multiplying each DCV by each BVM to produce a set of PCA 
coefficient vectors for each face image. 

17. The system of Claim 16, wherein the neural network ensemble 
5 employed comprises a plurality of face recognition neural networks each of 
which has input and output units ^nd is dedicated to a particular pose range, 
said face recognition neural networks output units being in communication with 
input units of a fusing neural network which has at least enough output units to 
allow a different output to represent each person it is desired to identify at each 
10 of the pose ranges, and wherein the sub-module for training the neural network 
ensemble further comprises sub-modiles for: 

for each face recognitiori neural network, inputting, one at a time, 
each of the PCA coefficient vectors associated with the pose range of the face 
recognition neural network into the inpiljs of the network until the outputs of the 
15 network stabilize; 

initializing the fusing neural network for training; 
for each DCV, simultaneously inputting the PCA coefficient vectors 
generated from the DCV into the respective face recognition neural network 
associated the vector's particular pose range group until all the PCA coefficient 
20 vectors of every DCV have been input, andl repeating until the outputs of the 
fusing neural network stabilize; and | 

for each DCV, simultaneously inputting the PCA coefficient vectors 
generated from the DCV into the respective face recognition neural network 
associated the vector's particular pose range group and assigning the active 
25 output of the fusing neural network as corresponding to the particular person 
and pose associated with the model image u^ed to create the set of PCA 
coefficient vectors. 
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18. The system of Claim 17, whereirl the sub-module for employing the 
neural network ensemble to identify the person depicted in the input image face 
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region and the pose associated N/v^th the face of the identified person, comprises 
sub-modules for: 

preparing the face regibn extracted from an input image by 
normalizing and cropping the extracted regions, wherein said normalizing 
comprises resizing the extracted face legion to the same prescribed scale if not 
already at the prescribed scale and adjusting the region so that the eye locations 
of the depicted subject fall within a prescribed area, and wherein the cropping 
comprises eliminated unneeded portion^ of the image not specifically depicting 
part of the face of the subject; 

concatenating the prepared face region to create a DCV; 

multiplying the DCV by each BVM to produce a set of PCA 
coefficient vectors for the extracted face region; 

inputting each PCA coefficipnt vector in the set of PCA coefficient 
vectors into the respective face recogniti^ 
vector's particular pose range group; anc 

identifying the active unit of the output of the fusing neural network 
and designating the person and pose previously assigned to that unit as the 
person and pose associated with the e)dtracted face region. 



n neural network associated that 



20 19. The system of Claim 1 3, wherein the program module for 

designating the face region extracted jrrom the input image to be an unknown 

person comprises sub-modules for: 

training a neural netwdirk ensemble to identify the person 

associated with the input image fads region to be an unknown person if it does 
25 not match any of the face regions ^assigned to each pose range to a prescribed 

degree; and 

employing the neui^l network ensemble to identify the person 
associated with the input image/face region to be an unknown person if it does 
not match any of the face regiqps assigned to each pose range to a prescribed 
30 degree. 
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20. A computer-readable memory for use in identifying a person 
depicted in an input image, comprising: 

a computer-read^t^le storage medium; and 
5 a computer prograVn comprising program modules stored in the 

storage medium, wherein the storage medium is so configured by the computer 
program that it causes a computer to, 

input model iqpages, each of which depicts at least one 
person of known identity, 
10 locate and extftact regions within the model images, each of 

which depicts the face of a known person that it is desired to identify in the input 
image, 

determine the f ace pose for each of the face regions 
extracted from the model images, 
15 categorize eaci' face region by assigning each to one of a 

set of pose ranges into which its ass ociated face pose falls, 

capture said input image, 

locate and extrai :t regions within the input images, each of 
which depicts the face of a person, a id 
2 0 for each face region extracted from the input image, 



comparing the face region to the face 



identify the person depicted in the inf ut image face region. 



regions assigned to each pose range to 



21 . The computer-reatiable memory of Claim 20, wherein the program 
2 5 module for comparing compriseasub-modules for: 

assessing the degrap-of similarity between the input image face 
region and the face regions assignOTTto each pose range; 

ascertaining which face region assigned to the pose ranges 
exhibits the greatest degree of similarity to the face region extracted from the 
30 input image; and 



58 



designating the facei region extracted from the input image to be 
the person depicted in the face region ascertained to exhibit the greatest degree 
of similarity to the input image face\region. 



10 
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22. The computer-readable\memory of Claim 21 , wherein the program 
module for comparing further comprises a sub-module for specifying that the 
face region extracted from the input image has a pose falling within the pose 
range to which the face region ascertained to exhibit the greatest degree of 
similarity to the input image face regiors was assigned. 

23. The computer-readable memory of Claim 20, further comprising a 
program module for designating the face region extracted from the input image 
to be an unknown person If the degree of similarity between the input image face 
region and the face regions assigned tOyp^tvpose range do not exceed a 
prescribed threshold. 



24. The computer-readable memory of Claim 20, wherein the program 
module for comparing comprises sub-mopules for: 

training a neural network e nsemble to identify the person depicted 
in the input image face region and the pc se associated with the face of the 
identified person; and 

employing the neural network ensemble to identify the person 
depicted in the input image face region t nd the pose associated with the face of 
the identified person. 



-\ 25. The computer-reacrable memory of Claim 24, wherein the sub- 
module for training the neural nelyvork ensemble comprises a sub-module for 



preparing each face region extrac ted from said model images by normalizing 
and cropping the extracted regior s, wherein said normalizing comprises resizing 
30 each extracted face region to the same prescribed scale if not already at the 
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prescribed scale and adjusting aach region so that the eye locations of the 
depicted subject fall within the same prescribed area, and wherein said cropping 
comprises eliminating unneeded pprtions of the image not specifically depicting 
part of the face of the subject. 



26. The computer-readable memory of Clam 25, wherein the sub- 
module for training the neural network^nsemble further comprises sub-modules 
for: 

(a) selecting a previously unselected one of the set of pose 

10 ranges; 

(b) choosing a prescribed number of the prepared face images 
of each person being modeled which hav^ been assigned to the selected pose 
range; 

(c) concatenating each of the chosen prepared face images to 
1 5 create a respective dimensional column ve ctor (DCV) for each; 

(d) computing a covarianc b matrix from the DCVs; 

(e) calculating eigenvectois and corresponding eigenvalues 
from the covariance matrix; 

(f) ranking the eigenvalues in descending order; 
2 0 (g) identifying a prescribec number of the top eigenvalues; 

(h) using the eigenvectors corresponding to the identified 
eigenvalues to form the rows of a basis vec tor matrix (BVM) for the selected 
pose range; 

(i) repeating actions (a) through (h) for each remaining pose 

25 range. 

0) multiplying each DCV 
coefficient vectors for each face image. 



Dy each BVM to produce a set of PCA 



27. The computer-readable memory of Claim 26, wherein the neural 
30 network ensemble employed comprises a plurality of face recognition neural 



networks each of which has input and output units and is dedicated to a 
particular pose range, said faceyrecognition neural networks output units being 
in communication with input units of a fusing neural network which has at least 
enough output units to allow a different output to represent each person it is 
5 desired to identify at each of the pose ranges, and wherein the sub-module for 
training the neural network ensemble further comprises sub-modules for: 

for each face recognitiqn neural network, inputting, one at a time, 
each of the PCA coefficient vectors associated with the pose range of the face 
recognition neural network into the inp^uts of the network until the outputs of the 
10 network stabilize; 

initializing the fusing neurhl network for training; 
for each DCV, simultaneously inputting the PCA coefficient vectors 
generated from the DCV into the respecffive face recognition neural network 
associated the vector's particular pose range group until all the PCA coefficient 
15 vectors of every DCV have been input, aqd repeating until the outputs of the 
fusing neural network stabilize; and 

for each DCV, simultaneously inputting the PCA coefficient vectors 
generated from the DCV into the respective face recognition neural network 
associated the vector's particular pose range group and assigning the active 
2 0 output of the fusing neural network as coiresponding to the particular person 
and pose associated with the model image used to create the set of PCA 
coefficient vectors. 



28. The computer-readable jhemory of Claim 27, wherein the sub- 
25 module for employing the neural network ensemble to identify the person 

depicted in the input image face region and the pose associated with the face of 
the identified person, comprises sup-modules for: 

preparing the face region extracted from an input image by 
normalizing and cropping the extracted regions, wherein said normalizing 
30 comprises resizing the extracted face region to the same prescribed scale if not 
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already at the prescribed scale and adjusting the region so that the eye locations 
of the depicted subject fall within a prescribed area, and wherein the cropping 
comprises eliminated unneeded pprtions of the image not specifically depicting 
part of the face of the subject; 

concatenating the pr^ared face region to create a DCV; 

multiplying the DCV bv each BVM to produce a set of PCA 
coefficient vectors for the extracted face region; 

inputting each PCA coefficient vector in the set of PCA coefficient 
vectors into the respective face recoglbition neural network associated that 
vector's particular pose range group; and 

identifying the active unitlof the output of the fusing neural network 
and designating the person and pose previously assigned to that unit as the 
person and pose associated with the extracted face region. 



15 
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29. The computer-readable memory of Claim 23, wherein the program 
module for designating the face region ^rtracted from the input image to be an 



unknown person comprises sub-modules 



associated with the input image face regi 



for: 



training a neural network ensemble to identify the person 



►n to be an unknown person if it does 



not match any of the face regions assigned to each pose range to a prescribed 
degree; and 

employing the neural network ensemble to identify the person 
associated with the input image face region to be an unknown person if it does 
not match any of the face regions assigned \o each pose range to a prescribed 
degree. 
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30. The process of Claim 1 , wherein the comparing, ascertaining, 
designating and specifying process actionWomprises the actions of: 

training a network ensemblqu^entify the person associated with 
the characterized input image region and tneir face pose, wherein the network 
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ensemble comprises, a first stage having a plurality of classifiers each of which 
is dedicated to a particular poselrange and outputs a measure of the similarity 
indicative of the similarity between said characterized input image region and 
each of said model image charaGjterizations associated with the particular pose 
range of the classifier, and a neunal network as its second stage, said measure 
of similar of each classifier being m communication with inputs of the neural 
network which has at least enoughputputs to allow a different output to 
represent each person it is desired to identify at each of the pose ranges; and 

employing the networklensemble to identify the person associated 
with the characterized input image region. 



^>P">^^— -81 . The process of Claim 39, wherein the process action for training 
the newvork ensemble comprises an aoion of deriving each model image 
characterization from a set of model imagW of people, wherein each model 
15 image of the same person shows that persofi at a different face pose , said 
deriving action comprising: 

extracting the portion of each m^del image depicting a face; 
normalizing the extracted portionW each model image by resizing 
it to a prescribed scale if not already at the prescribed scale and adjusting the 
20 region so that the eye locations of the depicted j^ubject fall within a prescribed 
area; 

cropping the extracted portion of e^ch model image by eliminating 
unneeded portions of the image not specifically depicting part of the face of the 
subject to create a model face image; 
2 5 concatenating each of the model U ce images to create a 

respective model dimensional column vector (DCV) for each, 

categorizing the model DVCs by a ^signing each to one of a set of 
pose ranges into which its associated face pos€ falls; 
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inputting the model DvC of the each model face image falling in a 
particular pose range, one at a time, tp a pre-selected classifier dedicated to the 
particular pose range. 



5 32. The process of Claim 31 , wlaerein the process action of training the 

network ensemble further comprises the actions of: 

initializing the fusing neural network for training; 

simultaneously inputting the respective DVC of each model face 
image into all classifiers, until the DVC of every model image has been input, 
1 0 and repeating until the outputs of the neural network stabilize; and 

simultaneously inputting the respective DVC of each model face 
image into all classifiers, and assigning the active output the neural network as 
corresponding to the particular person and po^e associated with the model 
image used to create the DCV. 

15 

33. The process of Claim 32, whereiji the process action of employing 
the network ensemble to identify the person depicted in the input image face 
region, comprises the actions of: | 

preparing the face region extramed from an input image by 
2 0 normalizing and cropping the extracted regioris, wherein said normalizing 

comprises resizing the extracted face region to the same prescribed scale if not 
already at the prescribed scale and adjustidig the region so that the eye locations 
of the depicted subject fall within a prescribed area, and wherein the cropping 
comprises eliminated unneeded portions of the image not specifically depicting 
25 part of the face of the subject; / 

concatenating the preparep face region to create a DCV; 
inputting the DVC of the race region into all classifiers; and 
identifying the active outpfjt of the neural network and designating 
the person previously assigned to that[jnit as the person associated with the 
30 extracted face region. 
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34. The process of ClaiiTf\33, further comprising a process action of 
specifying that the person designate^ as associated with the extracted face 
region has the face pose previously assigned to the identified active output. 




35. A face recognition neural network ensemble for identifying a 
person depicted in an input image and a face pose range among a set of pose 
ranges into which the face of each identified person falls, comprising: 

a plurality efface recognition neural networks each of which has 



10 input and output units and each of which is dedicated to a particular pose range; 
and 



the output units of said face recognition neural networks and which has at least 
enough output units to allow a different output to represent each person it is 
15 desired to identify at each of the pose ranges; and wherein 



is desired to identify as depicted in the input image are respectively input into 
separate ones of the input units of the face recognition neural networks causing 
a single one of the output units of the fusing neural network to become active, 
20 thereby indicating the identity of the person whose face was depicted in the input 
image as well as the pose range associated with the pose of the depicted face. 

36. The neural network ensemble of Claim 35, wherein the number of 
input units of each face recognition neural network equals the number of image 

25 feature characterizations derived from the face of the person to be identified as 
depicted in the input image. 

37. The neural network ensemble of Claim 35, wherein the number of 
output units of each face recognition neural network at least equals the number 

30 of different people it is desired to identify in the input image. 



a fusing neural network whose inputs are in communication with 



image feature characterizations derived from the face of a person it 
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38. The neural network ensemble of Claim 35, wherein the output units 
of each face recognition neural network output real values ranging from 0 to 1 . 

5 39. The neural network ensemble of Claim 35, wherein the number of 

input units of the fusing network is equal to the number of face recognition 
neural networks multiplied by the number of output units of any one of the face 
recognition neural networks, and wherein the number of output units of the 
fusing network is equal to the number of its input units. 

10 

40. The neural network ensemble of Claim 35, wherein the output units 
of each face recognition neural network are binary in that a particular output unit 
is active whenever it has the largest output amongst all the output units thereby 
representing a 1 , and otherwise inactive thereby representing a 0. 

15 

41 . The neural network ensemble of Claim 37, wherein there is one 
output unit of each face recognition neural network in addition to the number 
required to equal the number of different people it is desired to identify in the 
input image, and wherein at least one of the output units of the fusing neural 

20 network represents a person of unknown identity and unknown face pose. 

42. A face recognition network ensemble for identifying a person 
depicted in an input image and a face pose range among a set of pose ranges 
into which the face of each identified person falls, comprising: 

25 a plurality of classifiers each of which has input and output units 

and each of which is dedicated to a particular pose range; and 

a fusing neural network whose inputs are in communication with 
the output units of said classifiers and which has at least enough output units to 
allow a different output to represent each person it is desired to identify at each 

30 of the pose ranges; and wherein 
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image feature characterizations derived from the face of a person it 
is desired to identify as depicted in the input image are respectively input into 
separate ones of the input units of the face recognition classifiers causing a 
single one of the output units of the fusing neural network to become active, 
5 thereby indicating the identity of the person whose face was depicted in the input 
image as well as the pose range associated with the pose of the depicted face. 
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